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Program Description^ 



Middle College High School 



Middle College High Schools are alternative high schools located 
on college campuses that aim to help at-risk students complete 
high school and encourage them to attend college. The four-year 
program offers a project-centered, interdisciplinary curriculum 
with an emphasis on team teaching, individualized attention, 
and the development of critical thinking skills. Students are also 
offered support services, including specialized counseling, peer 



support, and career experience opportunities. In recent years, 
some Middle College High Schools have converted to the Early 
College High School model, which offers students a five-year, 
accelerated course of study during which they can earn an asso- 
ciate degree or two years of college credits, in addition to a high 
school diploma. This review focuses only on the four-year Middle 
College High School model. 



Research one study of Middle College High School meets What Works 
Clearinghouse (WWC) evidence standards. This randomized 
controlled trial included 394 students in the Seattle Public 
Schools who were assigned to either an intervention group 
that was offered admission to the alternative high school or a 
control group that was not offered admission. Control group 
students were free to participate in other regular and alternative 
high schools operated by the school district, as well as General 



Educational Development (GED) programs. Most control group 
students participated in one of these other education options.® 
Based on this study, the WWC considers the extent of evidence 
for Middle College High School to be small for both the staying in 
school and the completing school domains. No studies that meet 
WWC evidence standards with or without reservations examined the 
effectiveness of Middle College High School in the progressing in 
school domain. 



Eff6CtiV6neSS Middle College High School was found to have no discernible effects on staying in school or completing school. 



Staying in schooi Progressing in schooi Compieting school 

Rating of effectiveness No discernible effects na No discernible effects 

Improvement index^ Average: -3 percentile points na Average: +2 percentile points 

na = not applicable 

1. This report has been updated to include reviews of nine studies that were not included in the earlier review of Middle College High School. Of the 
additional studies, eight were not within the scope of the protocol, and one was within the scope of the protocol but did not meet evidence standards. 

A complete list and disposition of all studies reviewed is provided in the references. 

2. The descriptive information for this program was obtained from a publicly-availabie source: the program’s website {http://www.mcnc.us, downloaded 
December 2008). The WWC requests developers to review the program description sections for accuracy from their perspective. Further verification of 
the accuracy of the descriptive information for this program is beyond the scope of this review. 

3. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 

4. These numbers show the average of student-level improvement indices for all findings across the study. 
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AbS6nC6 of conflict The Middle College High School study summarized in this interven- 
Of interest tion report was prepared by staff of Mathematica Poiicy Research, 
Inc. (MPR). Because the principai investigator for the WWC dropout 



prevention review is an MPR staff member as well as a study author, 
the study was rated by staff members from IGF International. 



Additional program 
information 



Developer and contact 

Information on the history of the Middle College High School 
model and current resources for program implementation are 
available from the Middle College National Consortium (MCNC). 
Address: Ted Killmer, Director of Communications, Middle Col- 
lege National Consortium, 47-09 30th Street, Suite 600, Long 
Island City, NY 11101. Email: tkillmer@mcnc.us. Web: http://www. 
mcnc.us. Telephone: (718) 361-1981. 

Scope of use 

The MCNC reports that, as of September 2008, the Middle 
College High School program was operating in 29 school 
districts in 12 states. 

Description of intervention 

Middle College High Schools are alternative high schools that 
operate as formal collaborations between local school districts 
and colleges. The schools, which offer regular high school 
diplomas, are small— with fewer than 100 students per grade— 
and are located on college campuses. Faculty and students 
have access to the college’s educational resources and facilities, 
and students can take college-level courses. Since 2002, some 
Middle College High Schools have converted to the Early College 
High School model, which offers students a five-year, acceler- 
ated course of study during which they can earn an associate 



degree or two years of college credits at no cost, in addition to 
a high school diploma. This review focuses only on the four-year 
Middle College High School model. 

The Middle College High School curriculum emphasizes devel- 
opment of critical thinking skills and connecting what is learned 
to real-world experiences. These schools typically offer career- 
oriented courses and internships. In addition, students often must 
complete a community service requirement to graduate. Classes 
are taught by high school teachers from the local school district. 
Faculty teach collaboratively and integrate material across disci- 
plines. Within team-taught classes, students often participate in 
collaborative learning groups. Student-to-staff ratios are substan- 
tially lower than in traditional high school programs, allowing for 
more individualized attention. Middle College High Schools often 
use alternative assessment strategies, such as portfolios and oral 
presentations. They emphasize democratic school governance 
and use school committees— including administrators, faculty, 
parents, students, and college and community representatives— 
to provide input and guidance on school operations. 

Cost® 

Researchers estimated the cost of Middle College High School 
in Seattle to be $1,093 a student per month of program participa- 
tion— about 50% higher than the cost of educating a student in a 
regular school within the district (estimated to be $734 a month). 



R6S6arch Fifteen studies reviewed by the WWC investigated the effects of 
Middle College High School. One study (Dynarski et al., 1998) 
is a randomized controlled trial that meets WWC evidence 
standards. The remaining 14 studies do not meet either WWC 
evidence standards or eligibility screens. 

The Dynarski et al. (1998) study of Middle College High School 
was part of a larger evaluation examining the effectiveness of 16 

5. 



middle school and high school dropout prevention programs. 

The Middle College High School study used a random assign- 
ment design and included 394 students who applied to attend 
the alternative high school, which was operated by Seattle Public 
Schools in cooperation with Seattle Central Community College. 
Study participants were generally older students— their average 
age was just under 18— who were over-age for grade or had 



See Rosenberg, L., & Hershey, A. (1995). The cost of dropout prevention programs. Princeton, NJ: Mathematica Policy Research, Inc. Costs have been 
converted to 2008 dollars using the Consumer Price Index. 
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Research (continued) dropped out of school, students assigned to the control group 
did not receive Middle College High School services; however, 
they were free to participate in other regular and alternative edu- 
cation programs in the community. Most of them participated in 
one of these other education options. Findings presented in this 
report were drawn from a follow-up survey administered about 
two years after random assignment.® 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or medium to large (see the What Works Clearinghouse 



Extent of Evidence Categorization Scheme). The extent of 
evidence takes into account the number of studies and the 
total sample size across the studies that meet WWC evidence 
standards with or without reservations.^ 

The WWC considers the extent of evidence for Middle College 
High School to be small for both staying in school and complet- 
ing school. No studies that meet WWC evidence standards with 
or without reservations examined the effectiveness of Middle 
College High School in the progressing in school domain. 



Effectiveness 



Findings 

The WWC review of interventions for dropout prevention addresses 
student outcomes in three domains: staying in school, progressing 
in school, and completing school. The Dynarski et al. (1998) study 
included in this report covers two domains: staying in school and 
completing school. The findings below present the authors’ estimates 
and WWC-calculated estimates of the size and the statistical signifi- 
cance of the effects of Middle College High School on students.® 
Staying in School. Dynarski et al. (1998) reported that by the end 
of the second year after random assignment, 36% of students in 
the Middle College High School group had dropped out of school, 
compared with 33% of control group students— a difference that 
was not statistically significant. In addition, this difference was not 
large enough to be considered substantively important based on 
WWC standards (that is, an effect size of at least 0.25). 



Completing School. Dynarski et al. (1998) found that 40% of stu- 
dents in the Middle College High School group had earned a high 
school diploma or GED certificate two years after random assign- 
ment, compared with 38% of control group students— a difference 
that was not statistically significant or substantively important.® 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effective- 
ness takes into account four factors: the quality of the research 
design, the statistical significance of the findings, the size of 
the difference between participants in the intervention and the 
comparison conditions, and the consistency in findings across 
studies (see the WWC Intervention Rating Scheme). 



6. An additional follow-up survey was conducted at the end of year three with an early cohort of study participants. Because of relatively low response rates to 
this survey, as well as evidence of substantial intervention-control differences in baseline characteristics among respondents, these third-year results were 
not used in the WWC rating of the effectiveness of Middle College High School. These results are summarized in Appendices A4.2 and A4.3. 

7. The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on 
the number and size of studies. Additional factors associated with a related concept— external validity, such as the students’ demographics and the 
types of settings in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating 
was determined for Middle College High School is in Appendix A6. 

8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate 
the statistical significance, see Technical Details of WWC-Conducted Computations. For the Middle College High School study summarized here, no 
corrections for clustering or multiple comparisons were needed. 

9. In addition, analysis of third-year survey data, available for an early cohort, indicates no statistically significant effect of the intervention on completing 
school after three years. However, these longer-term results suggest that Middle College High School may have shifted these completions toward 
receipt of regular high school diplomas and away from receipt of GED certificates. Appendix A4.3 presents these longer-term results. 
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The WWC found Middle 
College High School to have 
no discernible effects on 
either staying in school or 
completing school. 



References 



Improvement index 
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resents the difference between the percentile rank of the average 
student in the intervention condition versus the percentile rank of 
the average student in the comparison condition. Unlike the rating 
of effectiveness, the improvement index is based entirely on the 
size of the effect, regardless of the statistical significance of the 
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Appendix 



Appendix A1 


Study Characteristics: Dynarski, Gieason, Rangarajan, & Wood, 1998 (randomized controiied triai) 


Characteristic 


Description 


Study citation 


Dynarski, M., Gleason, R, Rangarajan, A., & Wood, R. (1998). Impacts of dropout prevention programs: Final report. A research report from the School Dropout Demonstration 
Assistance Program evaluation. Princeton, NJ: Mathematica Policy Research, Inc. 


Participants 


The Middle College High Sc/roo/ study used a randomized controlled trial research design. The original study sample of 516 students was comprised of fwo cohorts. Cohort 1 
included 199 students in the intervention group and 123 students in the control group who applied to attend the alternative high school at the beginning of fhe 1992-1993 school 
year. Cohort 2 included 123 students in the intervention group and 71 students in the control group who applied to attend the alternative high school at the beginning of fhe 
1993-1994 school year. 

Participanfs were, on average, 18 years old af fhe fime they applied to Middle College High School. About half of the sample was African-American; about one in five was Cauca- 
sian; about one in 12 was Hispanic; and slightly fewer than one in four was Asian or ofher ethnicities. Participants were evenly split between males and females. Most were behind 
grade level at baseline; two-thirds had had discipline problems at school in the past year; and just over half had dropped ouf of school in the past. 

Results summarized here are drawn from a follow-up survey adminisfered two years after random assignment; 244 intervention-group students and 150 control-group students 
responded, for response rafes of 76 and 77 percent, respectively.'' Researchers compared the baseline characteristics of the two research groups on 13 demographic, socio- 
economic, and school performance measures. A sfafistical tesf of fhe overall difference between the follow-up survey respondents in the two research groups on the full sef of 13 
baseline characteristics found that the groups were not significantly different. 

An additional follow-up survey was conducted three years after random assignment with cohort 1 only. Because of relafively low response rates as well as evidence of substantial 
intervention-control baseline differences among respondents, these third-year results were not used for the WWC effectiveness rating of Middle College High School. These longer- 
term results are summarized in Appendices A4.2 and A4.3. 


Setting 


The Middle College High School study was conducted at an alternative high school on the campus of fhe Seattle Central Community College in Seattle, Washington. 


Intervention 


Seattle’s Middle College High School, which opened in 1990 and continues to operate, targets students who are close to dropping out or have dropped out in the past. It is run as a 
collaboration between the Seattle Public Schools and the Seattle Central Community College. It offers students the opportunity to earn a high school diploma in a small alternative 
school located on the community college campus. The school emphasizes the development of critical thinking skills and focuses on experiential learning, internships, and support 
services. Services for students include individual counseling, peer support groups, attendance monitoring, and career awareness. In addition, students can take community college 
courses and use the college’s academic and sports facilities. Students are active in school governance and operations. They participate in screening new applicants, running 
assembly programs, and formulating school policies on attendance, discipline, and other issues. 

At the time it was evaluated, the school enrolled about 300 students and its core academic curriculum focused on two core modules: (1) math/science and (2) integrated humani- 
ties. Each module was taught by a team of two teachers, supported by two or three paid, in-class tutors. Within these team-taught classes, students frequently participated in col- 
laborative learning groups and worked on projects with other students. Each quarter, teachers developed a curriculum around a unifying theme, such as “rights and responsibilities’’ 
(Hershey, Adelman, & Murray, 1995). 




(continued, 
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Appendix A1 Study Characteristics: Dynarski, Gieason, Rangarajan, & Wood, 1998 (randomized controiied triai) (continued) 



Characteristic 


Description 


Comparison 


The control group did not receive Middle College High Sc/roo/ services; however, they were free to participate in other reguiar and alternative education programs in the community. 
Most controi group students participated in one of these other education options. During the first year after random assignment, control group members reported spending 63% of 
their time enrolled in a school or GED program, on average, compared with 60% for intervention students. According to student self-reports, over a third of the time control group 
members spent enrolled in school, they were attending one of Seattle's other alternative high schools. 


Primary outcomes 
and measurement 


Two relevant outcomes from the Middle College High School sXudy are included in this review: whether participants dropped out of school and whether they earned a high school 
diploma or GED certificate. Eor a more detailed description of these outcome measures, see Appendices A2.1 and A2.2. 


Staff/teacher training 


Middle College High Sc/joo/ teachers were regular high school teachers employed by the Seattle Public Schools. No additional information about their specific training was available. 



1 . Although the overall second-year response rates for the two cohorts combined were similar for intervention and control group students, there were substantial intervention-control differences 
in response rates within each cohort, particularly cohort 2. For cohort 1, the response rate for intervention group students was 82%, and for control group students, it was 75%. For cohort 2, the 
response rate for intervention group students was 65%, and for control group students, it was 82%. 
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Appendix A2.1 Outcome measures for the staying in schooi domain 



Outcome measure 


Description 


Dropped out 


Percentage of students who dropped out of school by the end of the second follow-up year. These self-reported data were collected from follow-up surveys. 



Appendix A2.2 Outcome measures for the compieting schooi domain 



Outcome measure 


Description 


Earned a high school 
diploma or GED 


Percentage of students who received a high school diploma or GED certificate by the end of the second follow-up year. These self-reported data were collected from 
follow-up surveys. 
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Appendix A3.1 Summary of study findings inciuded in the rating for the staying in schooi domain^ 









Authors’ findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean 

difference^ 
(Middle College 
High School 
- comparison) 


Statistical 

Effect significance'* 

size^ (at a = 0.05) 


Improvement 

index^ 



Dropped out (%) Cohorts 1 and 2 


394 


Dynarski et al., 1998 (randomized controlled trial)^ 

36 33 -3 


-0.08 


ns 


-3 


Oomain average for staying in school 






-0.08 


ns 


-3 



ns = not statistically significant 



1. This appendix reports second year follow-up findings considered for the effectiveness rating and the average improvement index for the staying in school domain. Third-year follow-up findings, 
available for cohort 1 only, are not included in these ratings but are reported in Appendix A4.2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. For the “dropped out” outcome, signs were reversed on the 
mean difference, effect size, and improvement index, since a reduction in dropping out is a favorable outcome. Means from Dynarski et al. (1998) are estimated using regression models that 
control for baseline characteristics. 

3. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations. 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple com- 
parisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate statistical significance, see Technical Details of 
WWC-Conducted Computations. In the case of Dynarski et al. (1998), no corrections for clustering or multiple comparisons were needed. 
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Appendix A3.2 Summary of study findings inciuded in the rating for the compieting schooi domain^ 









Authors’ findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean 

difference^ 
Middle College 
High School 
- comparison) 


Statistical 

Effect significance'* 

size^ (at a = 0.05) 


Improvement 

index^ 



Dynarski et al., 1998 (randomized controlled trial)^ 



Earned a high school Cohorts 

diploma or GED (%) 1 and 2 


394 


40 


38 


2 


0,05 


ns 


+2 


Oomain average for completing school 










0.05 


ns 


+2 



ns = not statistically significant 



1. This appendix reports second year follow-up findings considered for the effectiveness rating and the average improvement index for the completing school domain. Third-year follow-up find- 
ings, available for cohort 1 only, are not included in these ratings but are reported in Appendix A4.3. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regres- 
sion models that control for baseline characteristics. 

3. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations. 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple 
comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate statistical significance, see Technical Details of 
WWC-Conducted Computations. In the case of Dynarski et al. (1998), no corrections for clustering or multiple comparisons were needed. 
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Appendix A4.1 



Summary of additional findings for the completing school domain^ 









Authors’ findings from the study 












Mean outcome 


WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Mean 

difference^ 

Middle College (Middle College 

High School Comparison High School 

group group - comparison) 


Statistical 

Effect significance'* 

size^ (at a = 0.05) 


Improvement 

index^ 



Dynarski et al., 1998 (randomized controlled trial)^ 



Earned a high school 


Cohorts 


394 


21 


18 


diploma (%) 


1 and 2 








Earned a GED certificate (%) 


Cohorts 
1 and 2 


394 


18 


20 



ns = not statistically significant 



1 . This appendix presents findings for the intervention’s separate effects on high school diploma receipt and on GED certificate receipt. The intervention’s effect on the combined measure of high 
school diploma or GED receipt was used for determining the effectiveness rating and is presented in Appendix A3. 2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regres- 
sion models that control for baseline characteristics. 

3. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations. 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple 
comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate statistical significance, see Technical Details of 
WWC-Conducted Computations. In the case of Dynarski et al. (1998), no corrections for clustering were needed. 
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Appendix A4.2 Summary of longer-term subgroup findings for the staying in school domain^ 









Authors’ findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean 

difference^ 
(Middle College 
High School 
- comparison) 


Statistical 

Effect significance'* 

size^ (at a = 0.05) 


Improvement 

index^ 



Dynarski et al., 1998 (randomized controlled trial)^ 



Dropped out at end of year 3 (%) Cohort 1 only 217 31 38 7 0.19 ns +7 

ns = not statistically significant 

1 . This appendix presents the third-year follow-up findings for measures in the staying in school domain. These findings were not used for intervention rating purposes because the third-year sur- 
vey was administered only to cohort 1 and because the survey had a relatively low response rate (67%). Moreover, substantial baseline differences existed between the intervention and control 
group members who responded to the third-year survey. The intervention’s effect on staying in school was rated based on the second-year follow-up findings presented in Appendix A3.1. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. For the “dropped out” outcome, signs were reversed on the 
mean difference, effect size, and improvement index, since a reduction in dropping out is a favorable outcome. Means from Dynarski et al. (1998) are estimated using regression models that 
control for baseline characteristics. 

3. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations. 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and -r50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple 
comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate statistical significance, see Technical Details of 
WWC-Conducted Computations. In the case of Dynarski et al. (1998), no corrections for clustering were needed. 
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Appendix A4.3 Summary of longer-term follow-up findings for the completing school domain^ 









Authors’ findings from the study 














Mean outcome 




WWC calculations 




Outcome measure 


Study 

sample 


Sample size 
(students) 


Middle College 

High School Comparison 

group group 


Mean 

difference^ 
(Middle College 
High School 
- comparison) 


Statistical 

Effect significance'* 

size^ (at a = 0.05) 


Improvement 

index^ 



Dynarski et al., 1998 (randomized controlled trial)^ 



Earned a high school diploma 
by end of year 3 (%) 


Cohort 1 only 


217 


31 


23 


8 


0.25 


ns 


+10 


Earned a GED certificate by 
end ot year 3 (%) 


Cohort 1 only 


217 


24 


37 


-13 


-0.38 


Statistically 

significant 


-15 


Earned a high school diploma or 
GED by end ot year 3 (%) 


Cohort 1 only 


217 


55 


61 


-6 


-0.15 


ns 


-6 



ns = not statistically significant 



1. This appendix presents the third-year follow-up findings for measures in the completing school domain. These findings were not used for intervention rating purposes because the third-year 
survey was administered only to cohort 1 and because the survey had a relatively low response rate (67%). Moreover, substantial baseline differences existed between the intervention and 
control group members who responded to the third-year survey. The intervention’s effect on completing school was rated based on the second-year follow-up findings presented in 
Appendix A3. 2. 

2. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group. Means from Dynarski et al. (1998) are estimated using regres- 
sion models that control for baseline characteristics. 

3. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations. 

4. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

5. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. 
The improvement index can take on values between -50 and +50, with positive numbers denoting results favorable to the intervention group. 

6. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple 
comparisons. For an explanation about the clustering correction, see the WWC Tutorial on Mismatch. For the formulas the WWC used to calculate statistical significance, see Technical Details of 
WWC-Conducted Computations. In the case of Dynarski et al. (1998), no corrections for clustering were needed. 
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Appendix A5.1 Middle College High School rating for the staying in schooi domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentiaiiy positive, mixed, no discernible effects, potentiaiiy negative, or negativeJ 
For the outcome domain of staying in schooi, the WWC rated Middle College High School as having no discernibie effects. 

Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1; No studies showing a statisticaily significant or substantively important effect, either positive or negative. 

Met. No studies showed statisticaliy significant or substantively important positive or negative effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statisticaiiy significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed statisticaliy significant or substantively important positive effects. 

AND 

• Criterion 2: No studies showing statisticaiiy significant or substantively important negative effects. 

Met. No studies showed statisticaiiy significant or substantively important negative effects. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantively important positive effect. 

Not met. No studies showed statisticaiiy significant or substantively important positive effects. 

AND 

• Criterion 2: No studies showing a statisticaliy significant or substantiveiy important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statisticaiiy significant or substantively important positive effects. 

Met. No studies showed statisticaiiy significant or substantively important negative effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the foliowing criteria. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed statistically significant or substantively important effects, either positive or negative. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed statistically significant or substantively important effects. 

(continued) 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Intervention Rating Scheme. 
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Appendix A5.1 Middle College High School rating for the staying in schooi domain (continued) 



Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies showed statistically significant or substantively important negative effects. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important pos/f/Ve effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No studies showed statistically significant or substantively important positive effects. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed statistically significant or substantively important negative effects. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed statistically significant or substantively important positive effects. 
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Appendix A5.2 MiMIe College High School rating for the compieting schooi domain 

The WWC rates an intervention’s effects for a given outcome domain as positive, potentiaiiy positive, mixed, no discernible effects, potentiaiiy negative, or negativeJ 
For the outcome domain of completing school, the WWC rated Middle College High School as having no discernible effects. 



Rating received 

No discernible effects: No affirmative evidence of effects. 

• Criterion 1: No studies showing a statisticaily significant or substantively important effect, either positive or negative. 

Met. No studies showed statisticaliy significant or substantively important positive or negative effects. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statisticaiiy significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed statisticaliy significant or substantively important positive effects. 

AND 

• Criterion 2: No studies showing statisticaiiy significant or substantively important negative effects. 

Met. No studies showed statisticaiiy significant or substantively important negative effects. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantively important positive effect. 

Not met. No studies showed statisticaiiy significant or substantively important positive effects. 

AND 

• Criterion 2: No studies showing a statisticaliy significant or substantiveiy important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statisticaiiy significant or substantively important positive effects. 

Not met. No studies showed statisticaiiy significant or substantively important negative effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the foliowing criteria. 

• Criterion 1: At ieast one study showing a statisticaiiy significant or substantively important positive effect, and at least one study showing a statistically significant 
or substantively important negative effect, but no more such studies than the number showing a statistically significant or substantively important positive effect. 

Not met. No studies showed statistically significant or substantively important effects, either positive or negative. 

OR 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. No studies showed statistically significant or substantively important effects. 

(continued) 

1. For rating purposes, the WWC considers the statistical significance of individual outcomes and the domain-level effect. The WWC also considers the size of the domain-level effect for ratings of 
potentially positive or potentially negative effects. For a complete description, see the WWC Intervention Rating Scheme. 
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Appendix A5.2 Middle College High School rating for the compieting schooi domain (continued) 



Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At least one study showing a statistically significant or substantively important negative effect. 

Not met. No studies showed statistically significant or substantively important negative effects. 

AND 

• Criterion 2: No studies showing a statistically significant or substantively important pos/f/Ve effect, or more studies showing statistically significant or substantively 
important negative effects than showing statistically significant or substantively important positive effects. 

Met. No studies showed statistically significant or substantively important positive effects. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant negative effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No studies showed statistically significant or substantively important negative effects. 

AND 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. No studies showed statistically significant or substantively important positive effects. 
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Appendix A6 Extent of evidence by domain 



Outcome domain 


Number of studies 


Schools 


Sample size 

Students 


Extent of evidence^ 


Staying in school 


1 


>1 


394 


Small 


Progressing in school 


na 


na 


na 


na 


Completing school 


1 


>1 


394 


Small 



na = not applicable/not studied 

1 . A rating of “medium to iarge” requires at ieast two studies and two schoois across studies in one domain and a totai sampie size across studies of at ieast 350 students or 14 ciassrooms. Other- 
wise, the rating is “smaii.” 
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